Inconvenient samples: Modeling the effects of non-consent by coupling observational and experimental results

Biased sampling of participants presents a major limiting factor for the generalizability of findings from behavioral studies. This effect may be especially pronounced in developmental studies, where parents serve as both the primary environmental input and decide whether their child participates in a study. To estimate the effects of parental non-consent, we coupled naturalistic observations of parent-child interactions with a behavioral test. Results showed that one particular parenting practice, the tendency to use questions to teach, associated with both children’s behavior in the test and parents’ tendencies to participate. Exploiting these associations with a model-based multiple imputation, we estimated that the means of the consented and not-consented groups could differ as much as 0.2 standard deviations for five of the seven test measurements we used, and standard deviations are likely underestimated. These results suggest that ignoring the role of consent may lead to systematic biases when generalizing beyond lab samples.